Effects of Cohort Selection on the Results of Cost-Effectiveness Analysis of Disease-Modifying Drugs for Relapsing-Remitting Multiple Sclerosis

BACKGROUND: Decision-analytic cost-effectiveness models are used to determine the most cost-effective treatment option on the basis of the best available data. Guidelines for pharmacoeconomic model development indicate that models should be updated as new evidence becomes available. OBJECTIVES: To evaluate the appropriateness of the clinical data that were selected for Goldberg et al.s 2009 model of cost-effectiveness in multiple sclerosis and calculate results based on a revised cohort selection method for intramuscular (IM) interferon (IFN) beta-1a. METHODS: The original model compared cost per relapse avoided for IM IFN beta-1a, subcutaneous (SC) IFN beta-1a, IFN beta-1b, and glatiramer acetate (GA) based on intent-to-treat (ITT) results from clinical trials. However, due to lower-than-expected subject dropout rates, the IM IFN beta-1a trial had sufficient statistical power to be terminated early and was subsequently found to have met its primary endpoint, time to sustained 1.0-point Expanded Disability Status Scale progression. Within the all-patient (ITT) cohort (n=301), approximately 43% of patients were followed for less than 2 years; 172 patients were followed for 2 years or more. In contrast, the proportions of patients followed for at least 2 years in the clinical trials of IFN beta-1b, SC IFN beta-1a, and GA were 92%, 90%, and 86%, respectively. To test the impact of data selection on the cost-effectiveness model results, we recreated the original model using both the all-patient and 2-year cohorts from the IM IFN beta-1a pivotal trial. We then compared our results with those of the original model. RESULTS: In the original model, costs per relapse avoided were $141,721 for IM IFN beta-1a, $80,589 for SC IFN beta-1a, $87,061 for SC IFN beta-1b, and $88,310 for GA. In the reanalysis using the 2-year completer data for IM IFN beta-1a, costs per relapse avoided were $77,980 for IM IFN beta-1a, $80,121 for SC IFN beta-1a, $86,572 for IFN beta-1b, and $87,767 for GA. The cost per relapse avoided for IM IFN beta-1a was approximately 45% lower than in the original analysis, whereas the recreated results for the other 3 therapies differed from the original results by less than 1%. Sensitivity analyses showed that the recreated model was robust and that the rank order of cost-effectiveness results was unaffected by changes to any univariate parameter. CONCLUSIONS: The current study highlights the importance of data selection in cost-effectiveness analyses. After limiting the pivotal trial data for IM IFN beta-1a to patients followed for at least 2 years, we found that IM IFN beta-1a was more cost-effective than in the original analysis, while results for the other first-line disease-modifying drugs remained stable.


• The analysis by Goldberg et al. included a patient cohort for
IFNβ-1a IM in which approximately 43% of patients had received treatment for less than 2 years compared with 8%-14% of patients in the other treatment cohorts. The present study replicated the model by Goldberg et al. but applied it to patient cohorts that are more consistent with each other by using data from patients who had received IFNβ-1a IM for at least 2 years. • The revised economic model estimated costs per relapse avoided of $77,980 for IFNβ-1a IM, $80,121 for IFNβ-1a SC, $86,572 for IFNβ-1b SC, and $87,767 for glatiramer acetate. The results for IFNβ-1a SC, IFNβ-1b SC, and glatiramer acetate were consistent (within 0.6%) with the original analysis, whereas the results for IFNβ-1a IM were approximately 45% lower. • The change in the result for IFNβ-1a IM highlights the importance of careful consideration of data selection for input into models and heightens awareness that seemingly small inconsistencies may have a large impact on analysis results and conclusions.
(NIH). In 1993, it was determined that the patient dropout rate was considerably lower than expected, and the accrued patient population provided sufficient statistical power to terminate the study early at the recommendation of the drug and safety review committee. The study was subsequently found to meet the primary endpoint, time to progression of at least 1.0 point on the Expanded Disability Status Scale (EDSS) sustained for 6 months. As a result of the early termination, of the 301 patients in the intent-to-treat (ITT, "all-patient") cohort, only 172 (57.1%) were followed for 2 or more years (the "2-year" cohort). The remaining 129 (42.9%) patients were followed for less than 2 years, including 7 patients (4.4%) who were followed for less than 1 year, 1 of whom was followed for less than 6 months. In economic models, the inclusion of patients with less than 1 year of follow-up becomes problematic when annualizing relapse rates. In the MSCRG trial, annualized relapse rate (ARR) was calculated as the number of relapses divided by the number of months on study drug, multiplied by 12. For example, a patient on study drug for 9 months with 1 relapse would have an ARR of 1.3 (i.e., [1/9] × 12). If the same patient were on study drug for 2 years (24 months) with no further relapses, the ARR would fall to 0.5 (i.e., [1/24] × 12). Conversely, for a patient not receiving active therapy (i.e., in the placebo group), an opposite effect may be apparent. Using a similar example, a patient on placebo for 9 months with 1 relapse would have an ARR of 1.3. If the patient continued to receive placebo for 2 years and experienced an additional relapse (assuming that additional relapses over time are more likely in a nontreated patient), the ARR would decrease to a value of 1 (i.e., [2/24] × 12). Thus in this example, a 9-month analysis of placebo-treated and drugtreated patients would show no difference in ARRs, but a 2-year analysis would indicate that the active treatment reduced the observed ARR by 50%.
This effect may, in part, explain why in the MSCRG trial the ARR was reduced from 0.67 in the all-patient cohort to 0.61 in the 2-year cohort for the IM IFNβ-1a group and yet increased from 0.82 to 0.90 in the placebo group. As these results suggest, the IM IFNβ-1a patient cohort selected for analysis in the model could be expected to have an impact on the analytical outcome. Therefore, it seemed logical to include the cohort that allows the best like-with-like comparison with the other firstline injectable DMDs; 90%-92% of patients in the IFN studies and 86% of patients in the GA study were followed for at least 24 months. We suggest that the 2-year patient cohort from the IM IFNβ-1a study provides a more appropriate comparator group for cost-effectiveness analyses.
In order to assess the similarity of the all-patient and 2-year cohorts in the timing of beneficial effects, Jacobs et al. determined the percentage of patients with sustained progression onset occurring during year 1 and year 2. 16 This assessment showed that over the first 52 weeks of the MSCRG study, D ecision-analytic models are often used to determine the most cost-effective treatment option on the basis of the best available data, providing a useful tool for decision makers evaluating health care interventions. Accordingly, several best practice guidelines for the development of costeffectiveness models have been published. [1][2][3] These guidelines are intended to assist researchers in identifying and selecting the best available evidence to incorporate into models.
Numerous cost-effectiveness analyses have been published on multiple sclerosis (MS). [4][5][6][7][8][9][10][11] A study by Goldberg et al. that was published in JMCP in 2009 compared the cost-effectiveness of 4 injectable disease-modifying drugs (DMDs) used for the first-line treatment of relapsing-remitting multiple sclerosis (RRMS). 12 Such comparisons are relevant since it has been reported that in the United States up to one-quarter of the total all-cause health care costs in patients with MS can be attributable to injectable MS therapies. 13 The model compared the cost per relapse avoided for intramuscular (IM) interferon beta-1a (IFNβ-1a), subcutaneous (SC) IFNβ-1a, IFNβ-1b, and glatiramer acetate (GA). This outcome measure is particularly relevant in U.S. managed care settings due to the significant short-term impact that relapses can have on MS-related expenses. O'Brien et al. (2003) reported that the most severe relapses, defined as those requiring hospitalization, cost an average of $12,870 (at 2002 price levels). Even the mildest relapses requiring only physician care and medications to treat symptoms had an average cost of $243. 14 As pointed out by Hakim in a 2003 JMCP editorial, models should "never be regarded as complete, but should be repeatedly updated and/or replaced as new evidence becomes available regarding their structure or inputs." 15 He also commented, "When model predictions are evaluated, it is more important to focus on the modeled relationship between inputs and outputs than on the outputs only." Hakim's observations are consistent with practice guidelines for decision-analytic model development. 1 Taking these points into consideration, we reinvestigated the model used by Goldberg et al., 12 making 1 important substitution to the model input: the patient cohort selected for the IM IFNβ-1a group. Whereas the patient cohorts for SC IFNβ-1a, IFNβ-1b, and GA consisted primarily (86%-92%) of patients who had completed at least 2 years of therapy, the IM IFNβ-1a patient cohort included a much smaller proportion of patients (57%) who had received therapy for at least 2 years when the study was terminated.
In the original report of the Multiple Sclerosis Collaborative Research Group (MSCRG) trial, Jacobs et al. (1996) offered an explanation for the differing lengths of time on IM IFNβ-1a. 16 MSCRG, the pivotal phase 3 trial for IM IFNβ-1a, was a multicenter, double-blind, placebo-controlled study initiated in 1990 with funding from the National Institutes of Health there were only minor differences in sustained progression onset between the all-patient and 2-year cohorts in both the placebo group (22.0% vs. 21.8%, respectively, a relative difference of 0.9%) and the IM IFNβ-1a treatment group (12.5% vs. 12.9%, a relative difference of -3.2%). However, over the second 52-week period, the differences in the percentages of patients with sustained progression onset were considerably larger for both placebo (16.5% vs. 14.7%, a relative difference of 10.9%) and IM IFNβ-1a (10.8% vs. 9.5%, a relative difference of 12.0%). Although it is not possible to directly relate disease progression with relapse occurrence, these data are indicative of differences between the 2 patient cohorts that may contribute to differences in model outcomes.
The objective of the present study was to test the impact of selecting the 2-year cohort rather than the all-patient cohort for IM IFNβ-1a on the results of the original model.

■■ Methods
A full description of the original model has previously been published. 12 In summary, the model was developed from the perspective of health care payers in the United States. Direct costs and health outcomes were modeled over a 2-year time horizon. The model used ITT data for relapse rates and disease progression from the pivotal clinical studies of the 4 DMDs. [16][17][18][19][20][21] Resource use and costs were obtained from several sources, including the published literature, the Red Book, and expert opinion. The primary outcome measure was cost per relapse avoided.
We recreated the original model using the same data sources for cost and effectiveness but replaced the MSCRG all-patient cohort ARRs with the 2-year cohort ARRs for both IM IFNβ-1a (0.67 vs. 0.61, respectively) and placebo (0.82 vs. 0.90, respectively). Using these data, the relative risk reduction for IM IFNβ-1a as compared with placebo increased from 18.3% in the all-patient cohort (as used by Goldberg et al. in their original analysis) to 32.2% in the reanalysis. The models were estimated using Microsoft Excel (Microsoft Corporation, Redmond, WA).

■■ Results
A comparison of the results of the original analysis and the reanalysis is shown in Table 1. Ranked from most to least cost-effective, costs per relapse avoided in the recreated model using the revised inputs for IM IFNβ-1a were $77,980 for IM IFNβ-1a, $80,121 for SC IFNβ-1a, $86,572 for IFNβ-1b, and $87,767 for GA. For SC IFNβ-1a, SC IFNβ-1b, and GA, the costs per relapse avoided in the reanalysis were within approximately 0.6% of the results reported in the original model. The stability in these results for these DMDs confirms the accuracy of our replication of the original model. However, the cost per relapse avoided using the 2-year patient cohort for IM IFNβ-1a was 45% less than that originally reported. Univariate sensitivity analyses were conducted on key clinical (Table 2) and cost (Table 3)

Sensitivity Analyses of Reanalyzed Model: Clinical Parameters
in any of the sensitivity analyses altered the rank order of the cost-effectiveness of the therapies.

■■ Discussion
The objective of this research was to highlight the impact that varying input parameters can have on the output of a costeffectiveness model. It was not intended to "correct" the previously published model but to explore how our interpretation of published best practice guidelines 1-3 provide an alternative conclusion to those that have been previously reported. 12 Nor was it our purpose to expand on the original cost-effectiveness model; rather, our intention was to replicate as closely as possible the analysis of Goldberg et al. For example, although results from other studies, such as the Independent Comparison of Interferon (INCOMIN) 22 and Evidence for Interferon Doseresponse: European-North American Comparative Efficacy (EVIDENCE) trials, 23 may provide data on the relative efficacy of interferons in the treatment of MS, we did not set out to provide "definitive" answers as to the relative clinical efficacy of the compounds under study. Indeed, as was outlined in the editorial by Hakim, "models and their results should not be considered as claims about the facts or as predictions about the future." 15 Results of the reanalysis suggested that the original model by Goldberg et al. was particularly sensitive to the ARR or, more precisely, the reduction in the relative risk of relapse. Therefore, particular attention must be paid to the values for this parameter that are entered into the model. Given that 86%-92% of patients in the other treatment groups had received therapy for at least 2 years, the most appropriate IM IFNβ-1a cohort to use is, similarly, patients who had completed at least 2 years of therapy. Using this alternative patient cohort, we found that the cost-effectiveness of IM IFNβ-1a was improved compared with what the original model showed, whereas that of the other first-line DMDs remained stable.

Limitations
As this reanalysis utilized the same model as the original analysis, the limitations discussed in the original manuscript 12 apply equally to this study. Because the fundamental difference between the 2 analyses is the relative risk reduction for IM IFNβ-1a, a major limitation is the determination of an accurate estimate for this parameter. Although we considered data from the 2-year patient cohort most appropriate for inclusion in the model, it should be noted that studies that served as sources of input data for the original model employed strict ITT analysis. In the reanalyzed model estimated in the present study, the data for IM IFNβ-1a 16 were taken from a subsample that had completed 2 years of therapy, whereas the data for IFNβ-1b, 17 SC IFNβ-1a, 20 and GA 18 included subjects who had withdrawn from the studies. Specifically, 20 (8.1%) of 247 patients in the study of SC IFNβ-1b, 36 (9.7%) of 371 patients in the study of SC IFNβ-1a, and 36 (14.3%) of 251 patients in the study of GA did not complete 2 years of therapy but still contributed data to the model. We believe that these relatively low rates of withdrawal, particularly from the IFN studies, had a minimal impact on the analyses.

■■ Conclusions
The results of this analysis underscore the importance of data selection in cost-effectiveness models. This study suggests that in economic analyses of first-line injectable DMDs for the treatment of RRMS, using a different patient population can substantially change the results and conclusions of the model. Our reanalysis, using patient populations that are more mutually consistent, provides an alternative interpretation of previously published cost-effectiveness data reflecting the results for patients from the source pivotal trials who received at least 2 years of therapy with IFNβ-1a IM, IFNβ-1a SC, IFNβ-1b, or GA.  Concept and design, data collection, and data interpretation were the work of Becker with the assistance of Dembek. Becker wrote the manuscript assisted by Dembek and Hughes. The manuscript was revised by Becker with the assistance of Dembek and Hughes.